Pattern optimization when measuring depth to a surface using lens focusing

ABSTRACT

A pattern of light is projected upon a surface to be measured which may be devoid of surface detail. A sharply focused image of the surface provides distance discrimination. Although the projected pattern may be separate from the imaging optics, a common optics path removes distortion, provides maximum sensitivity and eliminates processing for misalignment between projector and imager. Sequential cross-correlation, synchronous detection or percent modulation processing methods may be readily implemented to develop three-dimensional coordinates relative to the sensor for all in-focus regions of the image. Refocusing the lens provides depth coverage. The amount of data can be increased by adapting the projected pattern to produce a maximum of detail in the direction of minimum rate of change of depth of the surface being measured.

BACKGROUND OF THE INVENTION

The present application is a continuation-in-part of the parentapplication Ser. No. 566,687 filed Dec. 29, 1983.

In any camera system, range to the object may be approximated forobjects closer than the hyperfocal distance by observing the sharpnessof focus of the image of the object (or a portion thereof) as the cameralens is moved or indexed across its focusing range. At the position ofsharpest focus the range of the object (or portion thereof) may be readdirectly from a calibrated focus adjustment. This can be done wheneverthere is sufficient detail on the object surface or at its boundary topermit the focusing to be readily apparent to the human eye or to aprocessor which can determine sharpness of focus based upon fine detail.Sharpness of focus is determined when the spatial high frequency contentof a region is maximized or when the percentage modulation at a regionis maximized, or both. This can be achieved via many well known digitalprocessing schemes which use the digitized output from a TV camera asinput or alternatively, via the use of instantaneous holographic fouriertransform techniques which instantly yield spatial frequencyinformation. However, both techniques fail when the object hasinsufficient surface detail or its edges are gently curving so as todefy the requirements for the presence of some innate structure uponwhich to judge proper focusing.

The methods and arrangements described heretofore are for obtainingthree-dimensional measurements of surfaces by observing the point atwhich a projected pattern was most sharply imaged. The projected patternprovides detail on a surface lacking sufficient surface detail todetermine when the surface lies at the exact distance to which theprojector and camera are focused. However, a surface may exhibit highcurvature in certain areas. If linear patterns are projected such thatthe curvature is in the direction perpendicular to the projected lines,and an insufficient number of lines are able to be focused, then thedistance measurement fails.

SUMMARY OF THE INVENTION

It is an object of the present invention to overcome the prior artdisadvantages.

More particularly, it is an object of the present invention to measurethe distance from a sensor to a surface which may be devoid of surfacedetail and provide the three-dimensional co-ordinates of points on thesurface relative to the sensor.

A further essential object of the present invention is to measure thedistance from a sensor to a surface with rapidly changing surface depthrelative to the sensor, and to provide the three-dimensionalco-ordinates of points on the surface relative to the sensor.

In keeping with this object, and with still others which will becomeapparent as the description proceeds, the invention comprises projectinga pattern of light (not necessarily restricted to visibleelectromagnetic radiation) onto the surface to be measured; receivingthe reflected light from the surface through the same lens used forprojection; imaging the reflected pattern on a light sensitive surface;reading out the pattern detected by the light sensitive surface;determining the location in the pattern that maximally correlates with areference pattern or provides maximum percentage modulation; andreporting the co-ordinates, relative to the sensor, associated with thatlocation within the pattern.

An alternate method of the invention separately projects a pattern oflight upon the subject surface with a large depth of focus.

A third method of the invention separately projects a pattern of lightupon the subject surface with a narrow depth of focus, the distance tofocus being controlled to match the imaging system focal distance.

The present invention comprises, furthermore, optimizing the light forexpected surface contours.

The invention will hereafter be described with reference to an exemplaryembodiment, as illustrated in the drawings. However, it is to beunderstood that this embodiment is illustrated and described for thepurpose of information only, and that nothing therein is to beconsidered limiting of any aspect of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1a is an optical schematic of a plan view of an exemplaryembodiment of the invention;

FIG. 1b is a side view of the embodiment of FIG. 1a;

FIG. 1c is an axial view of the embodiment of FIG. 1a;

FIG. 2 is a block diagram of the processing for an exemplary embodimentof the invention;

FIG. 3 illustrates an exemplary projection slide pattern and typicalimage received;

FIG. 4 is a block diagram of one method of processing the received imagesignal;

FIG. 5 illustrates typical voltage waveforms in processing the image;

FIG. 6 shows the unique dimensional mapping;

FIG. 7 shows a pattern applicable to the problem encountered with highsurface curvature or slope relative to the sensor, in accordance withthe present invention;

FIG. 8 shows an imaged pattern on a flat surface;

FIG. 9 shows an imaged pattern on a surface with high curvatureorthogonal to the pattern;

FIG. 10 shows an imaged pattern on a surface with high curvatureparallel to the pattern;

FIG. 11 shows a flat surface in a side view with the focal plane of theprojected pattern intersecting the surface;

FIG. 12 shows a side view of a surface with high curvature.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention provides a solution to the failure of prior artdevices to measure distance to objects lacking surface detail. Therequirement for object detail is obviated by projection of a lightpattern containing detail onto the object surface. FIG. 1b shows theaddition of projected pattern 125 to the prior art system that only usedmovable lens 13 with image sensing surface 17. The pattern 125 on slide127 to be projected is illuminated by light source 120 with condensinglens 19. Light passing through transparent portions of pattern 125 ispartially reflected towards lens 13 by beam splitter 121, the remainderof the light being absorbed by light absorbing surface 14. Lens 13focuses the image of pattern 125 at a known distance from lens 13. Anobject 11 at that distance with a sloping surface 12 facing the devicewill have a sharp image of pattern 125 irradiating its surface at thein-focus distance 123 and a blurred image at further distances 122 ornearer distances 124 as illustrated in FIG. 1a of the system. Reflectedlight from the pattern projected on surface 12 is then imaged by lens 13on light sensitive surface 17 of detection device 15. A portion of thelight is lost due to the beam splitter 121. An axial view (FIG. 1c) ofdetector 15 is shown with the received image 18.

It is not necessary to use a single lens 13 and beam splitter 121 toproject a pattern 125 upon object 11 though many important advantagesare lost. A separate projector working in conjunction with a prior artdevice can add detail to a featureless surface 12 and thus enable themeasurement of distances to points on its surface. The projector may befixed focus with a depth of field adequate to encompass the measurementrange of the distance measuring device or operate cooperatively with thedistance measuring device to provide a sharply focused projection imageat a distance equal to the in-focus distance of the measuring device.Axial motion 126 of lens 13 provides this simultaneous adjustment forboth pattern projection and image reception in the embodiment shown inFIGS. 1a to 1c. This greatly simplifies this simultaneous adjustment andprovides the increased accuracy (sensitivity) obtainable by using asharply focused projected pattern. This sensitivity of focusing isincreased because motion 126 of the lens 13 away from its true focussimultaneously defocuses the projection of detail 123 on the object 11and defocuses the imaging 18 of the object on the pickup device 15.Thus, the sensitivity is doubled. It should be noted that the opticaldistance of the projection slide 127 from the beam splitter reflectingsurface must exactly equal the optical distance from the beam splitterreflecting surface to the detection device surface 17 to ensure thatoptimum focus is achieved simultaneously for both the projection andimaging paths. Also, the internal optical elements, including theprojection slide 127, must be carefully coated and internal structurestreated to avoid internal reflections since internally scatteredprojector light reaching the pickup means 15 will degrade imagecontrast.

When using the described single lens system of FIG. 1, a processingadvantage is obtained because the process of correlation against a knownimage or alternatively, spatial synchronous detection may be used toprocess the received image. This occurs because of the reciprocityproperties of the optical path from pattern 125 through lens 13 toobject 11 and back through lens 13 to detection surface 17. For example,if a flat white screen is set perpendicular to the lens optical axis atthe in-focus distance 123, a sharp image of pattern 125 will appear onthe screen. Due to lens effects, however, the image will be slightlygeometrically distorted. However, since the image of the pattern 125 onthe screen is imaged back through the same lens onto the detectiondevice 15, it can be seen that the image on the detection surface 17must be an exact duplicate of the pattern 125 except for any lens 13defects that cause a blurring of the image. This occurs because sendingthe distorted image back through the same lens 13 cancels thedistortion. Thus, the image on surface 17 from a flat screen at distance123 is essentially identical to the projected pattern 125. For anyposition of lens 13 in the system's working range 126, the screen may beplaced at the corresponding in-focus distance 123 and the pattern imageonto the detection surface 17 would be invariant. This occurs becausethe magnification of the projection part of the system is equal to thedemagnification of the detection part of the system.

Therefore, the image at the detection surface 17 is invariant except forthe blur due to defocusing regardless of where an object is located inits field of view. For maximum distance sensitivity, the system isnormally used with the aperture as wide open as possible. The preferredprocessing required to find regions of sharp focus consists of lookingfor fixed known patterns in each region of interest using correlationagainst the known pattern for that region or synchronous detectionagainst the known pattern. Even if the object 11 reflecting surface 12is not uniformly reflective or featureless, the same processing may beused to assist in focus determination since the image on detectionsurface 17 will consist of the spatial product of the object surfacereflectance and the projected pattern. Thus, except for completelynon-reflective object regions, the projected pattern (as modulated byobject reflectance) will always appear at the detection surface 17whenever the region in question is in focus with respect to the currentlens 13 adjustment 126. The percent modulation found in each localregion may also be used to determine optimum focus.

The projected pattern 125 may consist of any pattern that hasmathematical properties providing ease of processing. Such patterns asdot arrays, horizontal bars, vertical bars, angled bars, concentriccircles, etc. may be employed.

Similarly, to avoid the contrast diminishing effects of ambient lightwhen using this process, it is anticipated that light source 120 will bea strobelight or a narrow band illumination device (e.g. a laser) withcorresponding bandpass filter 129 in the detection path. A shutter 128may also be used synchronized with light source 120.

Additionally, when a vidicon device which may introduce distortion inreadout is used as detector 15, it is expected that a reference patternwill be generated by placing a flat screen at the system's focaldistance 123 to derive the actual pattern required when performing thecross correlation or synchronous detection process.

FIG. 2 is a block diagram of one method of the processing required todetermine the distance from the sensor 21 to the object 27. Themeasurement system consists of sensor 21 which has been detailed inFIGS. 1a to 1c, an analog to digital converter 23, memories 24 and 25for the measured and reference scene data, computer 26 and controller29. A pattern is projected from sensor 21 onto object 27 and thereflected light converted to a video signal 22 by a TV camera performingthe detection function of sensor 21. A/D converter 23 digitizes thevideo signal which is stored in memory 24. During calibration, memory 25stores a reference pattern as previously described. Computer 26cross-correlates the data from memories 24 and 25 and provides output 28to report the three-dimensional location information of all in-focussurface areas on object 27. Computer 26 also directs controller 29 tofocus at a new distance to repeat the process until all surface areashave been measured. If percent modulation is used instead ofcross-correlation then memory 25 is not required.

A more detailed explanation will be given using an analog processingmethod as shown in FIG. 4. Although an analog processing scheme is shownit should be understood that all of the processing may as easily beperformed by computer 26 shown in FIG. 2. For the example given, thepattern 125 projected will be assumed to be horizontal bars as shown in(a) of FIG. 3. The small region 123 in focus on object 11 provides image18 on detector 15, shown enlarged in (b) of FIG. 3. Pattern (a) willhave a Video reference signal 41 that will look like waveform 52 in FIG.5. The reference signal 52 is shown normalized to zero voltage level 51.Video Object signal 42 will have a waveform 54 corresponding to thenarrow in-focus region 18 of the image and level 53 and 55 in theblurred, out of focus regions either side of 18. The processing shown inFIG. 4 is typical of both correlation and synchronous detection. Theregional cross correlation process 43 does not require sliding the twosignals relative to each other because the desired component of theinput signal 42 is known to be synchronized to the reference 41. Theproduct 44 of the signals will have a waveform 57 corresponding to thein-focus region 54 and waveforms 56 and 58 corresponding to regions 53and 55, respectively. Low pass filter 45 output 46 will reveal thelocation of the in-focus region 18 as an increased level 520 relative tosurrounding regions 59 and 521. A threshold circuit 47 can then makediscrete decisions 523 for in-focus and 522, 524 for out-of-focus asoutput 48. The time of decision 523 relates directly to the location 18on detector 16 which in turn, with the known position of lens 13 in itsworking range 126 and calibration data for the system, can provide thethree-dimensional co-ordinates of the in-focus regions.

As shown in FIG. 6, the calibrated volume 61 will have a unique in-focusdistance, X, measured from lens 63 at each specific location in itsrange of motion 626. Additionally each off-axis location such as A, B, Cand D map to unique locations on the imaging surface 67 at a, b, c and drespectively.

Alternate processing, percentage modulation not shown, would operate bycomputing for each region 100(Vp-Vm)/(Vp+Vm) where Vp is the peakvoltage of waveform 54 and Vm is the minimum voltage. This function isonly a maximum in regions where sharp focus exists.

If the illumination pulse of arbitrary length can be sharply terminatedand the detector shutter 128 opened several nanoseconds later, theintense reflections internal to device 21 and at distances closer thanthe object 11 can be eliminated from reaching the detector 15 andreducing the contrast between bright and dark portions of image 18 andbetween image 18 and surrounding out-of-focus images.

The preceding description presents in detail the methods of obtainingthree-dimensional measurements of points on surfaces lacking surfacedetail using lens focusing. Problems of surface curvature or sloperelative to the sensor were not discussed. FIGS. 7 to 12 illustrate theproblem encountered with high surface curvature or slope relative to thesensor. FIG. 7 shows a pattern suggested above and FIG. 8 shows theexpected image for the arrangement shown in 11. FIG. 11 shows a flatsurface 213 in a side view with the focal plane 214 of the projectedpattern intersecting the surface. The pattern lines that strike surface213 in the region near the intersection with focal plane 214 will be inrelatively sharp focus and are imaged as lines 211. The remainder of thepattern will be out of focus such as lines 212. As already describedabove, the in-focus lines can be automatically detected and related todistance from the sensor to provide the means of obtaining 3-Dmeasurements. However, if the number of pattern lines 211 in focus aretoo few as in FIG. 9, then the measurement will not be obtainedreliably. Light intensity may be brightest in the region of best focus,but reflectivity variations can confuse this criteria. The number oflines in focus 211 can become too small if surface 213 is at a steepangle or local surface curvature is very great such as ridge 215 ofsurface 213, as seen in side view of FIG. 12.

The present invention resolves this problem by providing patternsoptimized for obtaining data from such surfaces. For example, if theregion of high curvature (ridge 215) extends across surface 213 in adirection parallel to projected pattern 210, giving image 216, then byrotating pattern 210 ninety degrees, image 217 will be obtained. Thein-focus parts 218 of the projected lines will contain sufficientmodulation depth to provide the identifying characteristic needed forautomatic measurement.

Likewise, for any known surface type, optimum patterns can be projectedto assure reliable mensuration. For unknown surface types, variouspatterns and/or orientations can be used to obtain data on the surfaceindependent of orientation of the surface features.

The invention has been described and illustrated with reference to anexemplary embodiment. It is not to be considered limited thereto,inasmuch as all modifications and variations which might offerthemselves are intended to be encompassed within the scope of theappended claims.

What is claimed is:
 1. A method for obtaining three-dimensionalmeasurements on objects which may be devoid of surface detail but haveknown characteristics, comprising the steps of: projecting a pattern oflight with a projector from a light source, said pattern having arelatively high spatial frequency in direction of minimum distancegradient relative to the projector; imaging reflected light from saidpattern with a lens on a light sensitive detection surface; said imagebeing sharply focused within a narrow object plane; calibrating by acomputer the image of each unique location of the sharply focused narrowobject plane; storing the calibration data in a memory; adjusting saidobject plane focal distance by said lens to intersect the surface of anobject to be measured at an intersection, said intersection containingmore than one detectable point on said object; processing the receivedimage by said computer to determine in-focus regions; and convertingsaid in-focus region information into three-dimensional measurementinformation from said calibration data.
 2. A method of obtainingthree-dimensional measurements on objects which may be devoid of surfacedetail and have unknown characteristics, comprising the steps of:projecting a pattern of light with a projector from a light source, saidpattern having a relatively high spatial frequency in one direction;projecting said pattern in one direction; imaging reflected light fromsaid pattern with a lens on a light sensitive detection surface; saidimage being sharply focused within a narrow object plane; calibrating bya computer the image of each unique location of the sharply focusednarrow object plane; storing the calibration data in a memory; adjustingsaid object plane focal distance by said lens to intersect the surfaceof an object to be measured at an intersection, said intersectioncontaining more than one detectable point on said object; processing thereceived image by said computer to determine in-focus regions;converting said in-focus region information into three-dimensionalmeasurement information from said calibration data; and repeating saidsteps in other directions for acquiring data missed by directionsproducing unreliable data.
 3. Apparatus for obtaining three-dimensionalmeasurements on objects which may be devoid of surface detail but haveknown characteristics, comprising: projector means for projecting apattern of light from a light source, said pattern having a relativelyhigh spatial frequency in direction of minimum distance gradientrelative to the projector; lens means for imaging reflected light fromsaid pattern on a light sensitive detection surface; said image beingsharply focused within a narrow object plane; computer means forcalibrating the image of each unique location of the sharply focusednarrow object plane; memory means for storing the calibration data;control means for adjusting said object plane focal distance with saidlens means to intersect the surface of an object to be measured at anintersection, said intersection containing more than one detectablepoint on said object; computer means for processing the received imageto determine in-focus regions, said computer means converting saidin-focus region information into three-dimensional meansurement fromsaid calibration data.
 4. Apparatus for obtaining three-dimensionalmeasurements on objects which may be devoid of surface detail and haveunknown characteristics, comprising: projector means for projecting apattern of light with a projector from a light source, said patternhaving a relatively high spatial frequency in one direction; projectingsaid pattern in first one direction and then in other directions foracquiring data missed by directions producing unreliable data; lensmeans for imaging reflected light from said pattern on a light sensitivedetection surface; said image being sharply focused within a narrowobject plane; computer means for calibrating the image of each uniquelocation of the sharply focused narrow object plane; memory means forstoring the calibration data; control means for adjusting said objectplane focal distance with said lens means ot intersect the surface of anobject to be measured at an intersection, said intersection containingmore than one detectable point on said object; compute means forprocessing the received image to determine in-focus regions, saidcomputer means converting said in-focus region information intothree-dimensional measurement information from said calibration data.